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Using the nominal group technique to select the most appropriate topics 
for postgraduate research students' seminars 


Abstract 

In 2003; the Faculty of Health & Behavioural Sciences at our university offered a series of postgraduate 
research seminars. The series of seminars; selected by a three-person faculty team; received a mixed reaction - 
some seminars attracted a large proportion of the students whereas others were poorly attended. Thus; it was 
decided to continue the seminar series in 2004, but to undertake some formative evaluation to determine the 
students' preferences for the seminar topics. It was important to gain input from as many students as possible, 
and to allow maximum latitude for responses. It was decided to conduct a two-stage evaluation process: 
nominal groups with a subset of postgraduate research students; followed by a survey of all students based on 
the groups. It was apparent from the results of this study that the students' preferences for seminar content 
were quite dissimilar to those anticipated by academic staff. This study demonstrates the advantages and 
disadvantages of the nominal group technique in investigating information and skill needs among 
postgraduate research students. 
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Abstract 

In 2003, the Faculty of Health & Behavioural 
Sciences at our university offered a series of 
postgraduate research seminars. The series of 
seminars, selected by a three-person faculty team, 
received a mixed reaction - some seminars 
attracted a large proportion of the students whereas 
others were poorly attended. Thus, it was decided 
to continue the seminar series in 2004, but to 
undertake some formative evaluation to determine 
the students’ preferences for the seminar topics. It 
was important to gain input from as many students 
as possible, and to allow maximum latitude for 
responses. It was decided to conduct a two-stage 
evaluation process: nominal groups with a subset of 
postgraduate research students; followed by a 
survey of all students based on the groups. It was 
apparent from the results of this study that the 
students’ preferences for seminar content were 
quite dissimilar to those anticipated by academic 
staff. This study demonstrates the advantages and 
disadvantages of the nominal group technique in 
investigating information and skill needs among 
postgraduate research students. 
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Introduction 

In January 2003, it was decided to introduce a series of seminars for research students in the 
Faculty of Health & Behavioural Sciences (FHBS) at our university. The FHBS includes four 
departments: Biomedical Sciences, Nursing, Psychology, and Public Health. The series of 
seminars, selected by a three-person team, received a mixed reaction - some of the 
seminars attracted a large proportion of the students whereas others were poorly attended. 
Thus, it was decided to undertake a small-scale formative evaluation to determine the 
appropriate topics for the 2004 seminars. 

The 2003 seminars 

The team responsible for the development of the seminars included the author, the Dean of 
the Faculty, and the Head of the Department of Nursing. The seminar topics were chosen 
during a brainstorming session between the three team members following informal 
discussions with current research students and a review of seminars currently offered 
elsewhere in the university (to minimize duplication). Additionally, it was decided to include 
two open forum sessions where the research students could discuss any 

issues of interest to them. The 2003 seminar series is summarized in Table 1. 


Table 1: Health & Behavioural Sciences Postgraduate Research Seminars 2003 


19 March 

Your thesis - Planning your thesis research and making it a part of your 
life 

30 April 

Training and development - what’s available at UOW and how to find it 

28 May 

Critical appraisal (evaluating research articles) 

02 July 

Writing journal articles (& choosing target journals) 

06 August 

Forum - open to any issues students want to discuss (with faculty 
representative from each department) 

10 September 

Career planning - deciding where you want to go and knowing how to get 
there 

15 October 

Writing a conference abstract & presenting your research at a conference 

19 November 

Writing ethics applications 

17 December 

Writing fellowship applications 


The purpose of the seminars was two-fold: first, to provide students with additional 
knowledge and skills to assist them in obtaining their research degree; second, to provide 
them with an opportunity to interact and share ideas with other research students. An 
important component of the networking was seen to be interaction between students across 
the four departments and at varying stages of completion of the research degree. For this 
reason, it was decided to make the seminar topics generic to the FHBS with input from the 
four departments to ensure that department-specific issues were addressed. Wherever 
possible, this was achieved by having a senior faculty member from each department attend 
and participate in the session. 
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Attendance at the 2003 seminars was considerably lower than expected, averaging less than 
10 students per seminar, and there was considerable variation in the number of attendees 
across the different topics. For example, the session on training and development attracted 
only 5 students, but three times as many attended the session on thesis planning, and the 
session on writing journal articles was standing-room only. 

Planning for the 2004 seminars 

Importantly, the feedback from attendees at all of the 2003 sessions was extremely positive, 
with informal discussion with the students after each session confirming that they gained 
valuable information from attending. 

Therefore, it was decided to continue the seminar series in 2004, but to undertake some 
formative evaluation to determine the students’ preferences for the seminar topics. The 
logistics of obtaining the views of a diverse body of students, many of whom are part-time 
and study off-campus, led to the obvious conclusion that a survey was the only viable 
method for collecting the evaluation data. However, the concern was that in developing such 
a survey, the questions and response options would be limited by preconceived ideas of 
“appropriate” seminar topics. 

For this reason, it was decided to conduct a two-stage evaluation process. First, two nominal 
groups with a subset of postgraduate research students to develop the questions and 
response options for the survey. Second, a survey of all FHBS postgraduate research 
students. 

The Nominal Group Technique (NGT) 

The Nominal Group Technique (NGT) is designed to facilitate collaborative and democratic 
decision making (Delbecq Van de Ven & Gustafson 1975). In its “pure” form the NGT 
consists of six steps: 

1. Individual generation of ideas. 

2. Recording of all participants’ ideas (in a round-robin format). 

3. Group discussion of all generated ideas (to organize the list and remove duplications). 

4. Preliminary vote to select the most important ideas. 

5. Group discussion of the vote outcomes (including additions and further merging of 
overlaps). 

6. Final voting on the priority of items. 

However, as will be discussed in the following section, there are many variants on the NGT 
when it is used in practice. 

Use of the NGT in education research 

The NGT has been used in educational settings to investigate a wide range of topics, 
including characteristics of expert associate teachers (Boudreau 2000), homework 
communication strategies in remedial education (Nelson et al. 2002), primary teachers’ 
beliefs about families' competence to contribute knowledge to their child's education 
(Moseman, 2003), evaluation of one-to-one teaching in general practice (Duggan & Cox 
1999), selecting topics for a BSN leadership course for Registered Nurses (RNs) (Waddell & 
Stephens 2000), and occupational health nurses' perceptions of their education and training 
needs for the “new public health agenda” (Bamford & Warder 2001). 
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As suggested above, there are numerous variations in the application of the Nominal Group 
Technique in practice. 

The Nelson et al. (2002) study did not use the NGT to generate ideas, but rather to rank and 
evaluate the effectiveness and feasibility of 44 potential strategies generated in an earlier 
focus group study. As described by the authors, the NGT started at the third step (group 
discussion of each idea) and was preceded by all participants being provided with 
summaries of the previous studies (i.e., the list of potential strategies from which to choose). 
The result of the process in this case was agreement on 14 strategies (of the original 44) 
which were seen to be the most effective and feasible. 

The Duggan and Cox (1999) study combined NGT and focus group research to develop an 
evaluative tool for assessing the effectiveness of trainers of general practitioner registrars. 
The NGT commenced with individual generation of ideas, with trainers and registrars writing 
down the characteristics of good teaching sessions. The NGT was then modified to utilize a 
“snowball” technique (Duggan & Cox 1999), with participants discussing their lists in pairs 
and adjusting them accordingly, then the pairs combining into increasingly larger groups 
repeatedly undergoing the same process, and so on until the whole group combined with one 
list. Finally, each individual marked on their own copy of the comprehensive list the five 
characteristics they personally felt to be the most important. 

The Waddell and Stephens (2000) study used the NGT to involve RNs in the selection of 
topics to be covered in their course, following a similar process to Duggan and Cox (1999). 
In this study, the RNs were asked to individually list their three most important topics for the 
course, then to share their list in pairs, and again in a group of four, resulting in a composite 
list for each group of four RNs. These lists were then posted on a wall, and students were 
given coloured dots with which to “vote” for topics. The final stage was a group discussion to 
reach consensus on the final topics for the course. It is interesting that Waddell and 
Stephens (2000) emphasise that the key advantage of the NGT is the “incorporation of the 
ideas and desires of all the learners, regardless of their levels of assertiveness or 
extroversion,” yet students did not get the opportunity to present their ideas until after the 
four-person teams had reached “consensus” (which could potentially result in ideas from the 
less assertive students being excluded from the consensus lists). 

Advantages and disadvantages of using NGT in education research 

The primary advantage of the NGT over other strategies is the enhanced opportunity for all 
participants to contribute ideas and to minimize the domination of the process by more 
confident or outspoken individuals (Vella, Goldfrad, Rowan, Bion & Black 2000). Other 
advantages (Nelson, Jayanthi, Brittain, Epstein & Bursuck 2002; de Ruyter 1996; Brahm & 
Kleiner 1996; Fox 1989; Butler & Howell 1980; Moore 1987; Delbecq, Van de Ven & 
Gustafson 1975) include: 

• The generation of a greater number of ideas than other group processes. 

• The generation of more creative ideas than other group processes. 

• The ease of interpreting the results (as ideas are generated, voted on/ranked, and 
evaluated at the session itself). 

• A greater sense of accomplishment for members (as the results are available 
immediately after the session). 

• The minimal resource requirements (a venue, facilitator, whiteboard, paper and 
pens). 

• The comparatively efficient use of time. 
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However, it is important to recognise the disadvantages, or limitations, of nominal groups (de 
Ruyter 1996; Brahm & Kleiner 1996; Fox 1989; Butler & Howell 1980; Moore 1987; Delbecq, 
Van de Ven & Gustafson 1975). These include: 

• The limited number of topics and issues that can be covered (tend to be single-topic 
sessions). 

• The limitation of idea generation to the meeting itself (i.e., no opportunity for 
participants to think about the issue in depth and generate additional ideas in their 
own time. 

• The need for participants to feel comfortable with, and remain within, a very 
structured group process. 

• The lack of anonymity, which may limit participants’ willingness to express their 
views. 

• The necessity for all members to be capable of, and comfortable with, expressing 
their ideas in writing and then communicating them verbally to the group. 

• The time commitment required from participants, and the necessity for them to attend 
a specific location at a given time, which may limit participant numbers. 

• The lack of generalizability of the results to the wider population due to the specific 
characteristics of the participants (both in terms of who is nominated to attend, and 
who agrees to participate. 

• The limited nature of the data (i.e, in terms of number of respondents) often requires 
a follow-up survey or other quantitative methodology prior to making final decisions 
about an issue. 

Stage 1 - The nominal groups 

Method 

It was decided to conduct two nominal groups: one with students at the beginning of their 
degree, to determine their perceived information/development needs; and one with recently 
completed students to determine both the information/development experiences they had 
found most useful during their degree, and the information/development experiences they felt 
they had missed out on. 

Each of the four department heads was asked to nominate four students - two who had 
recently completed their research degree (or were about to submit) and two who had 
commenced their research degree in 2003. These 16 students were contacted by email to 
ask for their participation in what was described as a small-group discussion for the planning 
of the 2004 seminar series. Six of the eight students in each group agreed to attend the 
session; reasons for refusal were being overseas or otherwise unable to get to the campus 
(two students) and being too busy with the thesis submission process (two students). An 
additional student cancelled on the day of the session due to family illness, leaving a final 
sample of five early-stage and six late-stage students. All of the participating students were 
given a free movie pass as a thank you, but they were not told of this until after they had 
agreed to participate. 

In order to retain the integrity of the NGT process - particularly in relation to ensuring input 
from all of the participants - stage one was designed to closely follow the initial steps of the 
NGT process. The later stages were altered to incorporate the survey in stage two of the 
study. 
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On arrival at the venue, participants were welcomed and given the opportunity to introduce 
themselves to the group (i.e., their name, the department they were from, the degree they 
were undertaking, and how long they had been enrolled in their degree). 

This was followed by an informal discussion about the 2003 seminars, focusing on how (or 
whether) the participants were informed of the dates and topics for the seminars, and the 
logistics of the seminars (e.g. appropriateness of the timeslot chosen). The nominal groups 
process was then explained to the participants, and they were given the question to be 
considered: “What topics do you think should be included in the 2004 seminar series.” 
Participants were encouraged to think as broadly as possible about the information or skills 
they felt they needed to complete their degree, and to include topics which may not lend 
themselves to the current 90-minute seminar but could be provided by other means (e.g. 
longer sessions or externally-provided seminars). This was seen as important to ensure that 
topics of interest to the students were not excluded simply because they did not appear 
amenable to the 90-minute format. 

Individual generation of ideas - participants were provided with a sheet of paper and a pen 
and given 15 minutes to independently record all of the topics they would like to have 
included in the 2004 seminar series, or as an adjunct to the seminar series. 

Recording of all participants’ ideas - after the 15 minutes writing time, participants were 
invited to volunteer one item from their list in turn (i.e. in a round-robin format) until all of their 
ideas were listed on the whiteboard. 

Group discussion of all generated ideas - once all of the ideas were listed on the whiteboard, 
they were discussed by the group to organize the list and remove duplications. 

Preliminary vote to select the most important ideas - participants were provided with a sheet 
of paper and asked to list the three topics from the board which they thought were the most 
important. Group two (late-stage students) were asked to make two lists: one listing the 
three topics they thought would be most valuable to them now; and a second listing the three 
topics they thought would have been most valuable to them earlier in their research degree. 

Group discussions of the vote outcomes were not held, nor was there final voting on the 
priority of items, as the intent was to incorporate all of the items which received one or more 
votes in the survey to be distributed to all postgraduate research students (phase two). 

Results 

Session 1 - early-stage students: 

After removal of duplications and combination of overlaps, a total of 16 seminar topics were 
nominated by the group. The topics and the number of votes they received are listed in 
Table 2. 

Session 2 - late-stage students: 

After removal of duplications and combination of overlaps, a total of 22 seminar topics were 
nominated by the group. The topics and the number of votes they received, both for 
“sessions I would like now” and “session I wish I had at the beginning” are listed in Table 3. 
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Table 2: Topics and votes from Session 1 (early-stage students) 


How to work with your supervisor (stress, motivation, conflict) and how to get help if 
there’s a problem 

3 

Overview of a thesis - length, structure 

2 

Job opportunities 

2 

Advanced English skills for thesis writing 

2 

General introduction (e.g., email, SOLS) - including network opportunity 

1 

Funding sources - conferences, trips, etc. (inc. external funding) 

1 

Postgraduate funds - how much do 1 have and how much control do 1 have over how its 
spent? 

1 

How to apply for research grants 

1 

New and updated software 

1 

What is an ideal PhD and how do you achieve it? (inc. timing, management, 
publications) 

1 

Overall picture of the department and faculty (e.g., who to contact, how to get copy 
cards, etc.) 

- 

Writing skills - proposal, literature review 

- 

Experiences of late-stage postgrad students (inc. time management) 

- 

Library information 

- 

Getting extensions on scholarships 

- 

OH&S requirements for collection of data from subjects (e.g., sterilizing of equipment) 

- 

TOTAL 

15 
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Table 3: Topics and votes from Session 2 (late-stage students) 



EARLIER 

NOW 

Publishing in journals/academic writing 

- 

5 

Conference presentations 

- 

4 

How to start thesis writing (when, where, how) - inc. staying on track & 
writer’s block 

3 

3 

Jobs after your thesis 

1 

2 

Writing grant applications 

- 

2 

Project management (inc. setting milestones and measuring 
achievement) 

3 

1 

‘Selling your thesis’ (punchy writing) 

- 

1 

Supervisor-student relationships (working styles, etc.) 

3 

- 

Rights and obligations of PhD students (inc. resources) 

2 

- 

Conceptual frameworks for thesis 

1 

- 

Library skills - advanced database searching techniques 

1 

- 

Processes for thesis submission and examination 

1 

- 

Lifestyle/emotional management of PhD 

1 

- 

Software packages (what there is and how to use them) 

1 

- 

Stats consulting service - when can you see them & what can they do 

1 

- 

Purpose of a PhD - remind me why I’m doing this 

1 

- 

Time management 

- 

- 

Academic writing 

- 

- 

Intellectual property 

- 

- 

Data management - logistics (e.g., backups) and university requirements 

- 

- 

Stats refresher course 

- 

- 

Study participant management (recruitment, retention etc.) 

- 

- 

TOTAL 

19 

18 
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Stage 2 - The survey 

Method 

A questionnaire was designed, based on the outcomes of the nominal groups, to canvass the 
views of all HBS postgraduate research students. The questionnaire was posted on the 
faculty website, and emails were sent to all postgraduate research students asking them to 
take part in the survey. This email included detailed instructions for saving, completing, and 
returning the questionnaire. A reminder email was sent three days later. 

The questionnaire contained two main sections: 

Logistics of the seminars: 

• Preferred length for the seminars (response options: one hour; 1.5 hours; two hours; 
no preference) 

• Preferred distribution for the seminars (response options: session time only; in 
session and out of session; no preference) 

• Preferred time of day for the seminars (response options: lunchtime, 12.00 start; early 
afternoon, 2.30 start; late afternoon, 4.30 start; no preference) 

• Preferred day for the seminars (response options: Monday; Wednesday; Friday; no 
preference) 

Seminar topics: 

The seminar topics generated by the nominal groups (phase one) were listed, and 
respondents were asked to indicate their likelihood of attending each seminar (“assuming the 
seminars were held at a time suitable for you”). For ease of completion, the seminar topics 
were grouped into four categories, based on the discussion from the nominal groups: The 
research degree; Writing your thesis; Research skills; Administration and resources. 
Respondents were also invited to nominate any additional seminar topics they would like to 
see offered which were not included in the list. 

Results 

A total of 25 usable surveys were returned. Eleven of these were from students in the 
Biomedical Sciences department; with the remainder from Public Health (7), Psychology (6), 
and Nursing (1). Fourteen of the respondents (56%) were enrolled full-time and 11 part-time; 
22 (88%) were enrolled as domestic students and three as overseas students. The majority 
of the respondents (64%) had been enrolled for less than three years, based on year of first 
enrolment (i.e., six enrolled in 2003, five in 2002, five in 2001, three in 2000, four in 1999, 
one in 1998, and one in 1993 with an extended leave of absence). Sixteen of the 
respondents (64%) had completed their undergraduate studies at the University of 
Wollongong, five at another Australian university, and four overseas. 

Logistics of the seminars 

Length: There was a clear preference for short seminar length, with 14 respondents (56%) 
expressing a preference for one-hour seminars, six for 90 minutes, one for two hours, and 
the remainder no preference. 
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Distribution-. Opinion was divided on whether the seminars should be held only during 
session (six respondents), or in and out of session (eight respondents) - with eleven 
respondents (44%) stating that they had no preference either way. 

Time: Opinion was also divided in relation to the best time of day for the seminars. 
Approximately even numbers of respondents stated a preference for seminars to be held at 
12.30 (eight), 2.30 (seven), and 4.30pm (seven), with the remaining three not expressing a 
preference. As would be expected, part-time students (who are generally employed) were 
more likely to express a preference for the later afternoon seminars (six part-timers versus 
one full-timer). 

Day: One-third (eight) of the respondents stated that they would prefer the seminars to be 
held on Wednesdays (as they were in 2003), six that they prefer Fridays, and 11 had no 
preference. Interestingly, none of the respondents expressed a preference for Monday 
seminars. 

Seminar topics 

For each of the listed topics respondents were asked whether they would “definitely attend,” 
“probably attend,” “possibly attend,” or “definitely not attend.” Table 4 lists the seminar topics 
in order of popularity (defined as the greatest proportion of students expressing an intention 
to definitely attend). A mean score for the total sample was calculated for each topic (shown 
in column 2), where 3 equals “definitely attend” and 0 equals “definitely not attend.” Column 
three shows the actual number of respondents who stated that they would definitely attend 
each of the seminars. As can be seen from the table, these two measures result in a similar 
prioritization of topics, although there are some small differences due to the distribution 
across the other response options. 

There were a total of six topics for which more than half of the respondents reported that they 
would definitely attend a seminar. These were: process for submission & examination (17); 
writing skills (selling thesis) (16); overview of thesis structure (15); how to start thesis writing 
(14); what is an ideal PhD (14); and writing grant applications (14). Importantly, the 2003 
seminar series included only one of these topics (writing grant applications). There were a 
further four topics for which ten or more (i.e., 40% or more) of the respondents said they 
would definitely attend: developing conceptual framework (12); job opportunities & career 
planning (11); publishing in journals (11); and time management (10). Two of these four 
topics were included in the 2003 seminar series (career planning and publishing in journals). 
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Table 4: Preferences for the seminar topics 



Mean 

Would definitely 
attend 

N (%) 

process for submission & examination 

2.6 

17 (68%) 

overview of thesis structure 

2.6 

15 (60%) 

how to start thesis writing 

2.4 

14 (56%) 

writing skills (selling thesis) 

2.4 

16 (64%) 

developing conceptual framework 

2.3 

12 (48%) 

what is an ideal PhD 

2.2 

14 (56%) 

job opportunities & career planning 

2.2 

11 (44%) 

publishing in journals 

2.2 

11 (44%) 

time management 

2.1 

10 (40%) 

writing grant applications 

2.1 

14 (56%) 

project management 

2.0 

8 (32%) 

how to work with your supervisor 

2.0 

9 (36%) 

statistics refresher course 

2.0 

9 (36%) 

conference presentations 

1.8 

8 (32%) 

software packages 

1.8 

7 (28%) 

intellectual property 

1.7 

5 (20%) 

lifestyle/emotional management of PhD 

1.6 

5 (20%) 

advanced library skills 

1.6 

6 (24%) 

data management logistics 

1.5 

2 (8%) 

rights & obligations of PhD students 

1.4 

5 (20%) 

statistical consulting service 

1.4 

5 (20%) 

study participant management 

1.3 

1 (4%) 

picture of dept, and faculty 

0.7 

2 (8%) 

thesis writing for NESB 

0.6 

4 (16%) 

general introduction 

0.5 

1 (4%) 


The following section reviews the responses to the proposed seminar topics by the four 
broad focii under which the seminars were categorized (the seminar titles and categories are 
exactly as they appeared on the survey form). Given the small sample sizes, it was not 
possible to conduct any meaningful statistical analyses of differences between sub-groups of 
respondents. However, respondents were dichotimised into: new (commenced 2003) versus 
old (commenced prior to 2003); and UOW undergraduate versus non-UOW undergraduate 
degree. Where the data identified apparent differences between the sub-groups, these are 
discussed below. 
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1. The research degree 

Four of the seven topics in this group were included in the respondents’ top ten. 

What is an ideal PhD and how do you achieve it? (inc. duration, time management, 
publications): Fourteen (56%) of the respondents reported that they would definitely attend 
this seminar, and a further five that they would probably attend. Only two respondents 
reported that they would definitely not attend a seminar on this topic. All six of the “new” 
students reported that they would definitely attend (compared to eight of the 19 “old” 
students). 

Time management (inc. experiences of late-stage postgrad students): Ten (40%) of the 
respondents reported that they would definitely attend this seminar, and a further nine that 
they would probably attend. Only one respondent reported that they would definitely not 
attend a seminar on this topic. 

Processes for thesis submission and examination: Seventeen (68%) of the respondents 
reported that they would definitely attend this seminar, and a further six that they would 
probably attend. None of the respondents reported that they would definitely not attend a 
seminar on this topic. 

Job opportunities and career planning: Eleven (44%) of the respondents reported that 
they would definitely attend this seminar, and a further nine that they would probably attend. 
Again, none of the respondents reported that they would definitely not attend a seminar on 
this topic. Five of the six “new” students reported that they would definitely attend (compared 
to six of the 19 “old” students). 

2. Writing your thesis 

Four of the five topics in this group were included in the respondents’ top ten. 

Overview of a thesis - length, structure: Fifteen (60%) of the respondents reported that 
they would definitely attend this seminar, and a further nine that they would probably attend. 
None of the respondents reported that they would definitely not attend a seminar on this 
topic. 

Developing/choosing a conceptual framework for your thesis: Twelve (48%) of the 
respondents reported that they would definitely attend this seminar, and a further eight that 
they would probably attend. Only two respondents reported that they would definitely not 
attend a seminar on this topic. 

How to start thesis writing (when, where, how) - inc. staying on track & writer’s block: 

Fourteen (56%) of the respondents reported that they would definitely attend this seminar, 
and a further five that they would probably attend. None of the respondents reported that 
they would definitely not attend a seminar on this topic. 

Writing skills (proposal, literature review, selling your thesis with ‘punchy’ writing): 

Sixteen (64%) of the respondents reported that they would definitely attend this seminar, and 
a further four that they would probably attend. Only two respondents reported that they 
would definitely not attend a seminar on this topic. 

3. Research skills 

Two of the seven topics in this group were included in the respondents’ top ten. 

Publishing in journals (inc. how is journal writing different to thesis writing): 11 (44%) of the 
respondents reported that they would definitely attend this seminar, and a further nine that 
they would probably attend. Only two respondents reported that they would definitely not 
attend a seminar on this topic. 
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Writing research grant applications: 14 (56%) of the respondents reported that they would 
definitely attend this seminar, and a further two that they would probably attend. Only two 
respondents reported that they would definitely not attend a seminar on this topic. 

4. Administration & resources 

None of the topics in this group were included in the respondents’ top ten. 

Discussion 

It is apparent from the results of this study that the students’ preferences for seminar content 
are quite dissimilar to those anticipated by the development team. Only three of the 
students’ top ten topics had been offered in the 2003 seminar series and a number of the 
preferred topics were outside the content areas that were under consideration for the 2004 
series. It is evident from the disparities between the list generated by academic staff (i.e., 
the 2003 seminar series) and the list generated by the students, that staff are perhaps not as 
aware of the students’ needs and preferences as was previously thought. 

There are several possible explanations for these disparities and it is important to recognise 
these in making decisions about the seminar topics to be addressed. The first explanation is 
that the development team was incorrect in their assumptions about the topics that would be 
of value to students - a reasonable supposition given that the team’s views were based on 
their own experiences and feedback from the research students they had supervised in their 
own areas of expertise. The second explanation is that the students are not aware of the 
gaps in their knowledge and the areas in which they need assistance - again a reasonable 
supposition as it is common ground that people often “don’t know what they don’t know.” 
The truth is probably a combination of these two explanations, and thus decisions about 
future seminar topics need to incorporate both the students’ perspectives of their needs and 
interests and the academics’ perspectives on the knowledge and skill gaps of the institution’s 
research students. 

As a result of this process, a series of recommendations were made - and subsequently 
acted upon - in relation to the seminar logistics and topics. As the students were deemed to 
be the best judges of their availability to attend the sessions on specific days and times, the 
2004 seminars were scheduled in line with the students’ responses. In terms of the specific 
topics to be covered, a list was developed based on the survey responses, feedback from 
previous seminars (particularly attendance levels), and consultation with academic staff. 

The ability to draw conclusions from this process reflects the advantages and disadvantages 
of nominal group techniques discussed in the introduction to this paper. 

The use of NGT enabled us to generate a greater number of ideas for potential seminar 
topics than were previously considered, and the topics were considerably more diverse than 
had been anticipated. Further, the final stage of the NGT (voting on ideas and collating the 
votes) enabled us to end the session with a comprehensive, but not overly long, list of 
seminar topics for inclusion in the survey. Informal feedback from the participants 
subsequent to the session confirmed that they found the process interesting and were 
satisfied with the outcomes and with their feelings of involvement in the process. 

However, the limits on the number of issues that can be covered meant that the study was 
able to focus only on the seminar topics themselves, and not on other issues that may 
influence students’ participation in the seminar series. It is acknowledged that there are a 
number of other factors that could influence students’ attendance at the seminars that were 
not specifically addressed in this study. 
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These include such things as distance from campus and ease of transport to the seminars; 
career and family responsibilities; perceptions of “belonging” to the Faculty as opposed to 
identifying solely with the department; and existing attitudes about the value of attending 
groups seminars. 

Other limitations of the study due to the nature of NGT research include: 

• The lack of opportunity for participants to think at length about knowledge and skills 
they would like to develop in the context of their degree due to the need to provide 
immediate feedback at the session. 

• The possibility that participants may not have expressed all of their perceived needs 
due to the lack of anonymity or to any discomfort with the formal group process. 

• The potential for some participants’ responses to be limited by their ability to express 
their thoughts in writing and/or verbally (particularly some of the overseas students). 

• The lack of generalizability of the results to the wider student population as the 
participants were nominated by their department heads (thus possible selection bias) 
and attended voluntarily (thus potential response bias). 

In conclusion, this study demonstrates the value and the limitations of the nominal group 
technique in investigating information and skill needs among postgraduate research 
students. The use of the nominal group technique enabled us to obtain an extensive and 
varied list of seminar topics - which we were then able to use as the basis for a survey of 
postgraduate students. Further, through this process it became evident that steps needed 
be taken to address the needs of a sub-group of students for information on the remaining 
topics: including providing students with information on where such seminars are offered 
outside of the faculty series (e.g., the library, the Student Learning Centre); and ensuring that 
written information is provided in the student handbooks. However, a number of questions 
were left unanswered by the process, including other reasons for (non) attendance, attitudes 
towards the seminars in general, and other options for delivery of research skills training. 
Future research could address these issues, as well as surveying students who have 
completed degrees to obtain retrospective opinions on the value of seminar topics. 

References 

Bamford, M & Warder, J (2001). Occupational health nurses' perceptions of their education 
and training needs to meet the new public health agenda using the nominal group 
technique, International Journal of Lifelong Education 20(4), 314-25. 

Brahm, C & Kleiner BH (1996). Advantages and disadvantages of group decision-making 
approaches, Team Performance Management: An International Journal 2(1), 30-35. 

Boudreau, P (2000). L’expertise d’un enseignant associe, McGill Journal of Education 35(1), 
53-70. [English abstract only] 

Butler, LM & Howell, RE (1980). Coping with Growth: Community Needs Assessment 
Techniques. Washington State University. WREP 44. 

de Ruyter, K (1996). Focus versus nominal group interviews: a comparative analysis, 
Marketing Intelligence & Planning 14(6), 44-50. 

Delbecq, AL, Van de Ven, AH & Gustafson, D (1975). Group techniques for program 
planning: A guide to nominal group and Delphi process. Glenview IL: Scott Foresman. 


Journal of University Teaching and Learning Practice - Vol. 1/1, 20-34, 2004 




Using the Nominal Group Technique to Select the Most Appropriate Topics for 

Postgraduate Research Student’s Seminars 

Sandra C. Jones 

Duggan, S & Cox, J (1999). Evaluating one-to-one teaching in general practice, Medical 
Teacher 21(1), 61-66. 

Fox, D (1989). The improved Nominal Group Technique (INGT), Journal of Management 
Development 8, 20-27. 

Moore, CM (1987). Group Techniques for idea building. Newbury Park, CA: Sage. 

Moseman, CM (2003). Primary Teachers Beliefs about Family Competence to Influence 
Classroom Practices, Early Education and Development 14(2), 125-153. 

Nelson, JS, Jayanthi, M, Brittain, CS, Epstein, MH & Bursuck, WD (2002). Using the Nominal 
Group Technique for homework communication decisions, Remedial and Special 
Education , 23(6), 379-386. 

Vella, K, Goldfrad, C, Rowan, K, Bion, J & Black, N (2000). Use of consensus development 
to establish national research priorities in critical care, British Medical Journal 320(7240), 
976-980. 

Waddell, DL & Stephens, S (2000). Use of learning contracts in a RN-to-BSN leadership 
course, The Journal of Continuing Education in Nursing 31(4), 179-184. 


Journal of University Teaching and Learning Practice - Vol. 1/1, 20-34, 2004 




